Skip to content

feat(node): producer-side batch accumulator - #544

Merged
kartikeya-27 merged 6 commits into
ByteVeda:masterfrom
stromanni:feat/node-batch-accumulator
Jul 25, 2026
Merged

feat(node): producer-side batch accumulator#544
kartikeya-27 merged 6 commits into
ByteVeda:masterfrom
stromanni:feat/node-batch-accumulator

Conversation

@stromanni

@stromanni stromanni commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Closes #516.

Node had enqueueMany but no accumulator that flushes on size or interval — Python has BatchAccumulator, Java has Batcher<T>. This adds Batcher plus a queue.batcher(name, options?) factory.

using batcher = queue.batcher("ingestLog", { maxSize: 100, maxWaitMs: 500, onError });
batcher.add([line], { priority: 5 });  // returns ids only when this call crossed maxSize
batcher.flush();                       // force
batcher.close();                       // flush remainder + stop the timer (also Symbol.dispose)

Which shape

Java's, not Python's. Java and Node buffer enqueues and flush N jobs through enqueueMany; Python coalesces buffered items into one job whose payload is the list, which needs a worker-side batch-handler contract (per-item results, partial-failure retry) that Node doesn't have. The issue framed the gap as an accumulator over enqueueMany. Python's @task(batch=) coalescing stays Python-only and is worth its own issue.

One place it goes further than Java: add(args, options?) keeps per-entry EnqueueOptions, because Node's enqueueMany takes options per entry where Java's takes one shared set.

Failure and lifecycle

  • A failed flush keeps its payloads. flush() splices the buffer before calling enqueueMany — a job.enqueued handler runs synchronously inside it and can re-enter add — and on throw puts the entries back at the head, so submission order holds.
  • A caller-triggered flush rethrows. A timed flush has no caller, so it reports to onError (or logs) and re-arms; otherwise entries would strand until the next add.
  • The timer is unref'd like every other timer in the SDK, so a partial buffer never holds the process open — and is lost on a bare exit. close() / using is the documented shutdown path.
  • Tunables validate to RangeError; add after close throws QueueError.

Docs

New node/api-reference/batching.mdx and node/guides/core/batching.mdx. The API page also documents enqueueMany, which Node had shipped undocumented, and the guide mirrors Java's producer-batching vs dequeue-batchSize framing. The Node block in shared/guides/core/tasks.mdx pointed only at enqueue-options; it now links the new guide.

Testing

12 tests in test/core/batcher.test.ts: size flush, timed flush, close/idempotence, Symbol.dispose, per-entry options, end-to-end run, failed flush retains entries, onError plus timed retry, tunable validation, argument typing, and job.enqueued re-entrancy.

Full Node suite 511 passed / 88 skipped; the 10 test/dashboard/ failures are pre-existing locally (no vite in the dashboard/ workspace). Typecheck and biome clean; docs check:parity and docs build clean.

Summary by CodeRabbit

  • New Features

    • Added Node SDK support for producer-side batching with enqueueMany and the Batcher helper.
    • Batches flush automatically when they reach a size limit or wait time, with support for retries and error handling.
    • Added scoped cleanup through close() and using/Symbol.dispose.
  • Documentation

    • Added API reference and core guide documentation covering producer and worker batching, configuration, failure handling, and shutdown behavior.
    • Updated navigation and the unreleased changelog.

Buffers enqueues for one task and flushes them through enqueueMany on a size
or time trigger, matching Java's Batcher and Python's BatchAccumulator.
Also documents enqueueMany, which Node shipped undocumented.
@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@stromanni, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 47 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: aaefe8dc-e62c-41ee-ba28-4497af9af6d1

📥 Commits

Reviewing files that changed from the base of the PR and between 43ae170 and 89393e8.

📒 Files selected for processing (3)
  • docs/content/docs/node/api-reference/batching.mdx
  • sdks/node/src/batching.ts
  • sdks/node/test/core/batcher.test.ts
📝 Walkthrough

Walkthrough

Adds a Node.js Batcher API that buffers producer enqueues and flushes them through enqueueMany by size, time, explicit calls, or disposal. It exposes the API through Queue, adds comprehensive tests, and documents the new batching behavior.

Changes

Node producer batching

Layer / File(s) Summary
Batcher buffering and flush lifecycle
sdks/node/src/batching.ts
Adds configurable buffering, size/time-triggered flushing, failure retention and retry, idempotent closing, and Symbol.dispose support.
Queue API exposure and behavioral validation
sdks/node/src/queue.ts, sdks/node/src/index.ts, sdks/node/test/core/batcher.test.ts
Exposes Queue.batcher() and package exports, with tests covering flushing, typing, shutdown, failures, retries, and re-entrant additions.
Batching API reference
docs/content/docs/node/api-reference/batching.mdx, docs/content/docs/node/api-reference/index.mdx, docs/content/docs/node/api-reference/meta.json
Documents enqueueMany, Batcher methods and state, failure handling, and timer/disposal behavior.
Batching guides and release references
docs/content/docs/node/guides/core/*, docs/content/docs/shared/guides/core/tasks.mdx, docs/content/docs/resources/changelog.mdx, CHANGELOG.md
Adds batching guidance, navigation, cross-links, and changelog entries for producer and worker batching.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Producer
  participant Queue
  participant Batcher
  participant Storage
  Producer->>Queue: batcher(task, options)
  Queue->>Batcher: construct batcher
  Producer->>Batcher: add(args, options)
  Batcher->>Batcher: buffer entries until threshold or timer
  Batcher->>Queue: enqueueMany(entries)
  Queue->>Storage: persist batch
  Storage-->>Queue: return job ids
  Queue-->>Batcher: return job ids
Loading

Possibly related PRs

Suggested reviewers: pratyush618

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding a producer-side batch accumulator for Node.
Linked Issues check ✅ Passed The PR implements the requested Node producer-side accumulator that flushes on size or interval, matching issue #516.
Out of Scope Changes check ✅ Passed The added docs, tests, and changelog entries support the batching feature and do not appear unrelated.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@sdks/node/src/batching.ts`:
- Around line 176-188: Update the error handling in the timer callback around
flush and the onError branch so an exception thrown by onError is caught and
contained, while still ensuring the existing re-arm logic runs for open batches
with buffered entries. Preserve the current warning behavior when no onError
handler is configured.
- Around line 79-82: Update the maxWaitMs validation in the batcher options
initialization to reject values above Node’s maximum supported timer delay,
2_147_483_647 milliseconds, in addition to the existing finite positive check.
Keep arm() passing the validated value to setTimeout and preserve the current
RangeError behavior and message context.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0c75ee3e-3e4f-4b49-9d66-b83159b45cb6

📥 Commits

Reviewing files that changed from the base of the PR and between 2266554 and 43ae170.

📒 Files selected for processing (13)
  • CHANGELOG.md
  • docs/content/docs/node/api-reference/batching.mdx
  • docs/content/docs/node/api-reference/index.mdx
  • docs/content/docs/node/api-reference/meta.json
  • docs/content/docs/node/guides/core/batching.mdx
  • docs/content/docs/node/guides/core/index.mdx
  • docs/content/docs/node/guides/core/meta.json
  • docs/content/docs/resources/changelog.mdx
  • docs/content/docs/shared/guides/core/tasks.mdx
  • sdks/node/src/batching.ts
  • sdks/node/src/index.ts
  • sdks/node/src/queue.ts
  • sdks/node/test/core/batcher.test.ts

Comment thread sdks/node/src/batching.ts
Comment thread sdks/node/src/batching.ts
Node clamps a setTimeout delay above 2147483647ms to 1ms, turning a long wait
into an immediate flush.
A handler that threw escaped the timer callback as an uncaught exception and
skipped the re-arm, stranding the buffered entries.
@kartikeya-27
kartikeya-27 merged commit ba8086d into ByteVeda:master Jul 25, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Node: producer-side batch accumulator

2 participants